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PREFACE 


Regression Analysis is an important branch of Applied Statistics 
dealing with Multifactor data. The beauty of the subject lies in 
developing a conceptually simple model expressing the relationship 
between a set of meaningful variables. Of late, the methodology of the 
subject has undergone considerable innovation by the application of 
generalized inverse(especially Moore and Penrose Generalized in- 
verse). Nevertheless, successful application of Regression Analysis 
requires an appreciation both theoretical as well as practical problerns 
dealing with live data. 


The main motivation of presenting this monograph is to fill up the need 
of the average graduate student pursuing a course in statistics covering 
this paper during a Semester (or a Trimester). There are books of highly 
exhaustive nature (like Draper & Smith Applied Regression Analysis, 
John Wiley & Sons) which may not exactly meet the requirements of 
students doing justice to the subject within the time constraint. Inthe — 
second place, readability and understanding from some of the ad- 
vanced texts in the subject is difficult for an average graduate student. 
Finally, a monograph like text dealing with the subject with major 
emphasis on generalized inverse following the line of approach of 
Linear Estimation Theory of late Prof. R. C. Bose (example, Deriva- 
tion of Normal Equations of Gauss Markov model using projections 
and by M.P. g-inverse) fills up a considerable gap in the present 
literature. Hence my endeavour for a readable book for the graduate 
and Post-graduate students in statistics. 


The first two chapters of the text are devoted to the development 
of linear Models and Testing of Linear Hypothesis in the classical 
manner: whereas, the third chapter is again a presentation of the 
Modelling and Analysis using generalized Inverse technique. The 
elementary basic results of g-inverse theory are already appended in 
the same chapter. Using the same techniques of solving most of the 
problems in Regression Analysis with ample illustrations a number of 
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: This has been developed by Rao and Mitra (1971), 
nsingularity oF SO. 

Boullion and. Ouell (177°), imation of Intra class cor 

Jogy relates to the estimation i 

Another aah ae . ear model for a two way clustered balanced 

relation coefficients ware d to any set up, say, nonorthogonal noncon- 

data, again, when 2 to MINIQUE Theory (Minimum Norm Quad- 

aah teeae Pe iniies developed by Rao and also by Chakrabarti 
ratic Un 


(1970) in respect of general analysis of designs. 
A single equation linear model is given by 


Max = Aaxm On xi + (Dn (1.1) 


$a matrix of known elements and U=(u, u, ... U,)’ 


ae un ts vector. 
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Q=(Y-A®y (Y~A@) 
=V'Y- @A’Y - YAO + 0440 
= V’Y-20A’Y + 9440 
§0/88=0 = -24’¥+24'4Q@=0 


(1.6) 


(by the rule of vector differentiation) 


=> A’AO = A’Y (known as normal equation) 

ais O=(A’Ay ary 

is called the least Squares estimate of © under G.M. set up. (C_R. Rao 
1983). 


1.2 GAUSS—AITKEN SET Up 
A more generalised set up due to Aitken is given by 
E(Y)=A (®) and D(U)=V (1.8) 
where D stands for dispersion matrix, 
Now choose an orthogonal matrix P 3 
PVP’ =A 


where A isa diagonal matrix since Vis 
the linear model (1.1) by A7'2P. the s 


(1.9) 


Positively definite. Premultiplying 
ame is transformed into 


(A"P) ¥=(A"P) A@ + A“? py (1.10) 
Now D(A"@PU)= A"P [Dp (Uy P A“? 
an A!?pyp A! 
=a" _ A Ay (1.11) 


Thus in the transformed model (1.10) since errors are uncorrelated [be- 
cause of (1.11)], itis brought to G.M. set up and the least squares estimate 
of ©, by replacing A by (A'pa) and Y to (A'?py in (1.7), is given by 


© = ((A"Pay (APayy! (A“2pay a-!py 
=[A'P AN? AP Ay! (4'P A“? apy) 
=[A’P’ A" PAY" (A’P’ A“'Pyy 


Best Linear Unbiased Estimator (BLUE) 


4’@ is estimable, then 3 a linear function C’Y 3 
rbOis ’ 


E(Cc’Y) =b’O 
C’A@ =O holding vO 
= , 
CA=pD 
= 
A’C=b 
=> 


Var (C’Y) = C? Var (¥;) + CP Var (¥2) + 1 + C? Var (¥,,) 
=Cf 0+ GQ o+...4+ C2 62 


n 
Pee @ = C-Co’} (1.20) 


Now in the class of all unbiased estimators our object is to find a 
Comes iter has the minimum sampling variance for b’O. Such 
: linear pina nction is called the Best Linear Unbiased Estimator (BLUE). 


(ha) 
nditional minimisation of 
? (1.22) 


J w.r.t. C, we have 
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ame le | 
Substituting in equation (1.22) we have 

C=A(A‘Ay'p (1.26) 
Provided (A A)” exists Further, if (A’A) is nonsingular, then from (1 26) 
we have 

C’CO =H (A‘Ay! Ag (A’Ay! bo 
= b' (A’AY! b (1.27) 
which is the variance of BLUE. But if (A’A) is singular, then a solution of 
r in (I .24) is given by 
A= (A’A) b (1.28) 


where (A’A) is a 8-inverse of A’A and (A’A) need not be unique. But 
whatever may be the solution of 1 we shall show that the solution of C 
will remain unique. To show this: 


Let A=TAj «xy iin wks) 


— (1.29) 
and assume without loss of generality that the first r columns are linearly 
independent and (m —r) columns are depending on the first r columns: 
then 


, 


p(A)=r 


Also note that [(A,’ 
A and 42) 


(1.30) 


rs r & submatrix of A of rank r exists. Let 
be two distinct solutions of (1.24) 


ie. AA =b and A’AX = 5 
=> AA (A ~ 1%) =9 (1.31) 
Consider [C — Cy [ce _¢®@>9 
where COSA and C® = 4. 
O=[AN AA? [4AM _ 4 12 
ee (a) i Ay A’A prt) ‘ee 12) (1.32) 


=0, by (131) 
fis (co e ce ic = Cc =0 
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models as (vey (VO)? of BLUE for an esti 
classic: Hence, varia 5 
pea 9 is given by : (1.41) 
itd b'O is give sista) Ib as 
constra ; 
ae 7 when (A’A) 1 singular 
equine Corollary: Nes ans 
time cc Variance of blue : ati — 
pe D(@)=(Ai' Ai) 
the bac © 
linear e where A = [0%), Og, --» On] 
Regres, Mw 
ot . ri . . 
a o,=| % | and (A’A) is singular. 
jj z 
sec Anji 
@ Fun 
‘ tee ' 1.4 ESTIMATION SPACE AND ERROR SPACE 
@ Misce 1 The vector space spanned by 04, O ..., Wp ‘3 denoted by Ma (a, Op .. 
ereoy at,,) and is known as the estimation space. A linear function /’Y is said to 
fa aoe i L belong to error space if 


ee l'AO=0V0 


tion L’Y to belong to 


al. The 


= A’A (0 _ 8% = 
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(Y-A ®Y (Y-A <9, say 


=(¥'-@' A’) (Y¥-A6) 


=(YY-8 4 ¥-vA64 Gy 


A @) 
50/80 =0 
= -2A'Y + 2A’ @ =O 
= A 


©=(A’A)y! a’y 


(1.43) 

provided A’A is nonsingular. Normal equations are given by 

A'Y=A’AQ (1.44) 
1.5.1 A Theorem on Normal Equations 
Normal equations are always consistent. 
Proof : 
Normal equations A’Y = A’A 6 are Consistent; because 

Pp (A’A) = p (A’A, A’Y) (1.45) 


Since A’Y is a linear combination of y 


ectors of A, (i.e. O), Ob, ..., 
©,,) the normal equations are 


always solvable. 


Remarks: However, if (A’A) is singular, the solution of © iS Not unique. 
But whatever solution of @ is taken A © will remain unique. 


To show this; let @” and Q”) be two solutions of ©: then 


A’'Y=A’A O" and A’Y =A’A O® 


(1.46) 
Next consider 


(AO 4% (46_ 46% >0 
In view of equation (1.46) 
ow ~ Q®y AYA eo me 6?) 7 
AO = 40. 


ee 


a 


1.6 GAUSS—MaRKov THEOREM 
(2) Under G.M. set up, if C’ 


Y is BLUE for 6’, then 
CY=6'6 
where @ is O.L.S.E. (Ordinary Least Squares Estim 
Oe APA SY (24 6); = (> NO=RSS. 
Proof (/) : 


ator). 


Case I: If C’y is unbiased for 6’, then 


E(C’Y) =e 
CA O='0, VE 
AC=p 
Let C=A A be the consistent solution of A’C 
OS SEE NP UA 


(1.47) 
Also A’Y =A’A © are the Normal Equations and the O.L.S.E. for 
©, say O satisfies 


=e 


A'Y=A'AO 
CVY=V A'VY=V AAO 
A’C=A'(AA)=b, 
b’=NA'A 
b’O=NV AAO 
CY=NVA'Y=V A'AO=N'O 


[Note that the relation C’Y = b’ considerably simplifies the computa- 
tion of BLUE of a linear parametric function]. 


With CY¥=b'0 

E(b’ 0) =E(CY)=CAQ=N AAO 
b'O=C’AO=N A'AO 
b’=CA=N AA 


(as C=A A) 


oe + : 
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N A’A Q 
E (b i EQ A YY = A‘ e 
which proves that b’ © is unbiased for be 
Also Var (b’ ©) = Var (c’ Y) 
= Cr oJ 
*N AAR (ae ¢ Ad 
= bh. o- 
VAr (D’ @) = b’ (A’A - oS 
if A’A is nonsingular. Again from (1.27 Vitriance ofa 
timable parametric function is given by p (A’A) 
gular. 
= Var (b’ ©) = Var (BI UE) = Var (C 
Case II : Also, if A’A 1S singular W in WTI 
Var (b’ ©) =b' (A’AY bo 
| { A,’ A.yY 


Var (b’ Q) = b,’ (A; A,\) bb & 


Also, from (1.27) we see that variance of BLUE of an estimable 
parametric function is given by the same 


i.e. by (Ay Ay) by oe 
-. Bb’ provides the BLUE. 


Proof (ii) : 


E[(Y-A®y (Y-A®)]=(n-)o 


We have, p (A)=r, E(Y)=A® and D(Yj=01 
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. re: 
af equations a as 
Norm A'Y=A‘A e 
@= (A’Ay A’Y is not unique 
> . 
But as shown earlier, 
A@=A(A‘A) AY 
remains invariant 
(A,’A,)" 0 A,’ 
: 0 0 A,’ 
i , -l ? 
AO=A (A;’A,)” A\'¥ (1.48) 
where (A;’A;) is non-singular symmetric matrix. Hence, there exists a 


non-singular matrix G such that 


GG =(A\'A,)" 3) 
oth (G can be obtained by Choleski’s method of factorisation [Froberg Car] 
« _ E (1981).] 

= Now — Aj (A)’ Aj)" Ay’ =A, G’ GA,’ 
@ Pre | | = (GA,')' (GA,’) 


pA, (A;’A\) Ay’] =p [A, G’ GA,’] 
=p [(GA,'Y GA,’)] 
=p (GAy’) 

=p ())=r 


) is also idempotent. Since, A, 
nt 
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Next, we require to Show that 


(Y-A®)=[1~4 


MAYA AVY1(Y¥Y~A@) 


rae. (1.50) 

= 4.6 ACR AF ca rare tear 
‘ i 
Now A © =A, (4, 4,y"! AVY, from (1.48) 


Also Aj (Ai’ A\)' AvA@ =A e 


= RAS. of(150)=y_4@_, O+A0 


=¥i46 
= LHS. of (1.50) 
Again (Y—A Oy (y_ 46) 


=(f-A @Y UA, 4 Ay A 


U-A\ yA) AYA 6) 


=(Y-A6y U-A\ Ay Ay'ay(y—4 Q) (1.51) 


E((Y-A 6y (Y-A®) 
=EIY—A Oy (I-A, (Ay Ay Ay} (y—A 8) 


R.HS. of above being a quadratic form, i.e. a scalar, by putting 
[I-A, (A;’A,)' A, ]=V= [V,], we get 
ELE {yi-~E OD Vit Z E((,-E0) 0,-EG)) Vil 
t#j 
second term of which is zero. Simplifying we have 


OE Vj=0" trace [1 — A, (Ay’ A," Ay'] 


t 
=o (n—r) 
Hence, the estimate of 0” is given by 


go _(/-A Oy (Y¥-A8) 


(1.52) 
(n-r) 


(w) 


A 4 A\ 
1B) = 41 | a, s 
var (BLUB) yay Ay bLY=A OY (Y-A 6) 
e b (n—1) 
Markov theorem rigourously. It follows tha, 
ves Gauss~ 
This prove? 


Bem 
o is estimated fro 


. ( i 
hich Is BLUE foro where r=P A 
Ww 


Example 1.2: Verify - 
pena LEm-LAe 
Var (L’Y) =L’D (Y) L 

aR EL 

Vel; if 2=01 


following : 


() 
(i!) 


eon L 
(ii) Cov (L'Y, MY) 
=E[{L'Y-ELY)) (MY-EM YY 
_E(L'Y-E(LY)) E(M'Y-E(M’Y)] 
= E{L’ (Y-A®)] [M’ (Y-A ©)]’ 
-E[L’ (Y-A®)] [M' (Y-A ®)] 
=L/E[(Y-A®@) (Y-A®)]'M 
=L/5M 
=L'o° IM: if [=o] 
=o l’M 
If(B), « (BYY),«; is same as k linear functions of Y 
E(BY)=BA@ 
D(BY) = BD (¥) aoe, 


| “BIB =o" BB. where 5 = 92/. 
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1.7 DISTRIBUTION oF Quapratic FORMS 
To derive the probability 
(Onx1 Where X isa random vector and 
with specified rank, say p (A)=r< 
Algebraic results. One such m 
which is stated as follows: 


dis i : 
tribution of Quadratic forms (X’) (A 
‘ lun V by my 
A is a matrix of Elven elements 


nm, itis hecessary to 


4 é know certain 
ajOr result is 


Known as Cochran’ s theorem 
Cochran’s Theorem 


If O;=X A,X(i= 1; 2, ..., k) be k quadr 


atic forms with p (Aj) = r, and if 
: Ee . « ne 
A;’s are positively semidefinite matrices. 


et SApeeG, i= 1,2, ... b andy 


Q= x Q;= X’X = 2 rm 


i=] ge} 


then a necessary and sufficient condition th 
transformation, transforming each of 


Q; = X’A; X; (i=1, 2, ...,k) to 


at a Common non-singular 


rr trt+. +r 
pas 2 
Q; ee x Lu 


@=7 ++... +r. ,+1 


i 


exists is that r) +r)+... + rp=n. 


Proof : The Condition is Sufficient 


We suppose that 
htmt+...¢R=n 


Since p (Aj) =r; i= 1, 2,..., k, there exists anon-singular transfor- 
mation 3 


P; A; P;=(,, Daxian 


where U,, 9)= 


0 zy 

0 22 

0 2n 
ey ra 
pee 0 a 
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=} 
(by assumption r, +r, +... +r, =n) 
Again, if X = PZ is the transformation, transforming 
Q=X’X into | 


ZPPZ= % 2 
a= | 


=Z'P’ 1, PZ: Z 


Further, } 

Q1+ r+... +O,=Z' [(P\’ Ay Py) + (Py Ay P2) +... + (Py A, P| Z 1 

=Z’ PI, PZ 4 

“ 

which holds V Z & 
=> P'1,P = Pj’ A, P, + Py Ay P) +... + Py A, P, 


=P’ (A, +A, +...+A,) P : 


n 
=PPias >= A;=I, (1.55) 
i=1 


The result holds if P, = P, =... = P,=P. 


Putting P;= P; (i= 1, 2, ..., k) in (1.53) and (1.54) and noting that 
ifr; +r. +...+7,=n, then 3 a common transformation X = PZ which 


transforms 


QO; =X’A\X=Z’' PA, PZ 
=Z' (I,,0)Z 


= E¢2 
a=1 
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a=r, tl 
Op =X’ AX=Z! PA, PZ 
=Z' (I, 0)Z 


and 


Gant. they tt 


The condition is Necessary 


We have ~ 
Fiat ote n 
Q= = Za = Zz zq=X’X 
a=1 a=1 


under a common non-singular transformation X = PZ 


If possible, let r, +r. +... +7 <n, then 


p(Q)<n 


which is contrary to our hypothesis p (Q)=n. On the other hand if 
1 +1) +...+%>n, then the transformation X = PZ is singular which is 


again in contradiction to the hypothesis. 


on of Cochran’s Theorem 


o-/,) and Q= X’X ~ x7 (a 
necessary and sufficient 


Fundamentals of Linear Estimation 
aa 


(a chisquare with n; degrees of freedom) with 


XZ PA)=rn+r4., 


pe -Frp=yR 


1.7.2 Another Important Result 


Theorem: A necessary and sufficient co 
expressed as r sum of squares, where r= 
formation X = PZ is that A is an idempot 


Proof : (i) The Condition is Necessary 


ndition that Q = X’AX 
P (A), by an orthog. 
€nt matrix. 


Can be 
onal trans- 


If X = PZ be an orthogonal transformation by which 
Q = X"AX ~ Z’ P’APZ 


he , | , 
=Z° P" APZ(p’ = Pp as P is orthogonal) 


meee 
a=] 
eee 
= P' AP =(I,,0)= 
0 0 
<6 
=> A=P Pp 
0 0 
a. I, a 
= A’=P P'p Pp! 
0-6 6 0] 
ee 
=p P'=A 
fey 


which shows A to be an idempotent matrix. 


__188625 
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‘AiG 


+= GAy Al 
4) (GA) = oe 
ey Ata Ay 


ys; y 


(2.11) 


(GAL exe (GArYanr™ hi 
(GA\)rx 
= (Eyn-r8" Jnxn 
imilar ariay like GA,’ 
trix where if represents a sim 1 of 
be gee ae and L is arbitrary which can be chosen in any 
ant 
wma of ways. Let us denote 
GAy’ A 
@,.a= (Y-A®), «1 
mx 
eee AS == 2 2.12) 


f=1 
‘Also, (Y-A @Y A; (Ay’ A, Ay’ (Y-A ©) 
=(Y-A @) A, G’ GA,’ (Y- A) 


Ra y-A @)J' [GA,’ (Y-A ©)] 
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2 . 
thus Z, ~ N (0, 0°); 
td : 2 
since RHS of (2.12)/o* = y? - 
RHS of (2.13)/o? = re 
f vie 
and RHS of (2.14)/o°= 7 _. 


and all are independently distributed 
BLUE of b’ O=c’y 


Also BLUE of 
b’ O= by (Ay’ Ay A\Y 
ade , , | 7, 
= b,’ (A, Ay)” A, (Y-A0)+b/ (A; A,y' AVA®@ 


= b;’ G’ GA,’ (Y-A®)+C’A®@ 
(because C’=b,’ (A) A\y' Ay’) 
But C’A=D’' by the condition of estimability 
b’ @=b' © + b,’ G’ GA,’ (Y-A®) 
=b'O+b,' G’Z, (2.15) 


A 
Distribution of b’ ©, which is linear combination of normal vari- 
ables, is normal with mean zero 


Now—b,'G’z,=1'e=b'6 
and under Hp, b’ © must be zero 
by G'Z,=v' 0 
C'Y=b' O=b/ GZ, 
= [by (Ay A) AVY) 
Var (C’Y) = Var (b’ ©) 
=b,’ (Ay Ai) bo 
by’ (Ay Ai) Ay Y (2.16) 


<< 
by’ (Ay Ay) 19 
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have also shown that the residual mean 
We 


dard normal variate: 
tan 


isas 3 
oda. Kar (A + Jae 6’ A’ 
dual Mean S-S _ yy ee” 
eee gt (n— 1) 
Le. (e) 
oy oat 


Hence, an appropriate test is 
, a | 
by’ (Ay Av Y/by' (A Ar) 1 
I 
f= : 
yY'Y-@0' A’Y/(n-1r) 


where ris distributed as Student’s ¢ with (n — r) df. 


ing company of electric bulb while test- 
Oe heeled 
eas from a manufactured lot. All the n bulbs were lighted for a period 
of thours; and the distribution of k < n bulbs which were exhausted durin g 
t hours are given in order as 44) < 2) < ... < tj. Assuming that the life 
times of bulbs are distributed independently and exponentially given by 
f()=he™. Obtain the BLUE of the mean life time of the bulbs and the 
standard error of BLUE. (88625 
Hints : Here, ty) <ty)<... <j are not independent. But r 
fy ~fq_1) are independent random vari 
given by 


(Kay — t(1), 
ables whose distributions are 


Ply ty >the ehe-r+ i [Biswas.S (1994)] 


On of of the form 
+... F Hi (fey — ta 1) 


ential distribution with 
mum i.e. unconditionally 
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—< 
Example 2.2: E(Y)=X © where y= x 
on U, the error vector. Is ©, the ordina 


O+U under no Specification 
Ty least Squares estimator, yp. 
biased? 


2.3 THIRD PROBLEM 


To test Hy: b O=0 against H, 


0 CaO Gey 7 
linear parametric functions simul 


taneously 

Suppose out of k linear parametric 
pendent and the remaining (k — k’) functi 
k’ functions. 


yk) ie, lesting k 


functions k’ (k’ < k) are inde- 


Ons are dependent on the first 


bi 
bin 
: (bi), | 
Let bj=| 5, |= , Say 
dirs | (bi)m—rx| 


BLUE of b’O=b,’ (A; A\)' A/Y=C’Y 
BLUE of b,’O=by,' (A\’A,)' A/’Y 
BLUE of by’ =by)' (A A,)'A/'Y 


BLUE of by = by,’ (A\’ A, Ay'Y 


. . . , -| 
-. BLUE of all the k independent parametric functions is B, (A;’ A) 


A,’Y, where 


Again we have, 


Var (b’ @) = by’ (Ay Ay)! bo 


. Ww v 
a 


a 
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ey Ai yap are 4 @) with r df. 
| nner“ ay 41, '-4© 
| yma e oy L ae 
with (2-1 OF a yey 
i In vieW of GAy’ (r= i. 


4 eee 
G’ and (A; Ad) 


12 (Y-AQY Ai 2 nea 
2/2 G’ GB, [B\GG' By} 'B, G’GA,’ (Y—A 0) 


gaa” 


acl a 
— .- form with rank =k and the matrix under the [ ] jp 
ich ji iu fic : 
eo saat and symmetric. 
4) 


Z= 2 , where z-N (0,0) 


<n 
Hence, Q is distributed as yo with k d.f. Therefore, to test 
Hy: b/ @=0; i=1,2,..,& we may employ F-test given by 


Asay 


THEORY OF LINEAR 
ESTIMATION BASED on 


3.1 CONCEPT OF A GENERALISED INVERSE 


Hf (A)mxn with p (A)=r<min (m,n), then A* 
m=nandr=n(orm),i.e.unlessA isa non-singular matrix. Accordingly, 
a generalised inverse (A*),,,.,, (or A_) is defined as 


G-INVERSE 


AA* A=] 


It can be shown that A® always exists but is not unique in general 
(Biswas S, 1993). The simplest form of generalised inverse (g-inverse) of 
any matrix A is to partition A as 


(Aj). x5 alex ai 
(Adnxn= 


(Ay)m —-rxr (Ag)(m —r)x(a-r) 


Ip (A) = rand if we assume without any loss of generality p (A\) = rthen 


A, being an r-square non-singular matrix Aj! exists. Under this situation 
one form of g-inverse is 


is not defined unless 


s for null matrices of appropriate order. One 


; Ce ur 
£4 =A. In fact any A* of the form eat verify in 


where 0's stand 


this case that AA 
ve U 
Af = Q P 
V W 
for arbitrary U, V and Wis a generalised inverse of A, where 
i, 0 
be all e 
0 0 


where P and Q are two non-singular matrices. This shows that a 
number of generalised inverses can be constructed corresponding to i 
ny 


matrix A with any rank and order. 


3.1.1 Special Forms of g-inverses 

(a) Reflexive generalised inverse : 
If (A’),, m is a matrix which satisfies the following conditions - 
(i) AATA=A 

and (fi) A’AA’=A’ 

then As alle the reflexive g-inverse of A. 
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a erse 


(c) Right Weak g-inverse - ithe 
1f A” is a matrix satisfying 
(i) AA"A=A ee 
(ii) A" AA" =A" 
made (i) (A" A) =A"A 


then A” is aright weak g-inverse. 


(d) Moore and Penrose (M.P) g-inverse: 


If (A’), x m Satisfies the following: 


(i) AATA=A 

(7) A’ AA*=A‘ 

(iii) (A*AY =A*A 
and (iv) (AA*) =AA* 


then A’ is called M.P. g-inverse 


It can be shown that A” is unique. 


3.1.2 Construction of Different Forms of 
g-inverses 
A’ (reflexive g-inverse) can be constructed as 
A’ = A®\A A® (3.1) 
where A*! and A®?are any two g-inverses of A and A® may be constructed 
as : i 
1 
A® =A’ (A’A)’ (3.2) 
where (A’A)’ is a reflexive g-inverse of A’A constructed as in GAy 
Similarly, A” may be constructed as 
A" =(A’A) A’ (3.3) 


Finally A* may be constructed as 
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r (3.4) 
Area A 


f A. 
and left weak g-inverses © 
t 
A” are ms 


=p A" 
(vi) | a “ rank and A is full row rank. 
provided B is full column 


NATIVE PROOF OF MARKOV’S THEOREM 
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a areata a CRONE OU th a 


a ce 
N id b’At y 


as A*Y is the L.S. solution of Y=A Q@+U 


We have 
E (bA* Y) = b’A* E(Y) 
=b’ AAO 
Again b’=C’A 
=> E(U(A'Y)=C’ AA“ AG 
=C’AO=)’0 


Next, we shall show that b’A* Y j 


S BLUE of B’© and it has the 
property of uniqueness. 


Let us consider the normal equation 
AY=A’AO 
The particular solution is given by 
© =(A’A)* A’Y=At y 


(3.5) 
and the general solution is given by 


O=(A' AY A'Y+[I,— (A’AY* (A’A)} Z 
where Z is an arbitrary column vector 
= b’ O=b' (A’ AY) A’Y +’ Um -(A’A) (A’AY]Z 3.6) 
Using b’ O=b' A*Y +b’ [,, -(A*A))Z 
=b' A*Y+b'Z-b’ (A*A)Z 
Putting b’ = C’A (condition of unbiasedness) 


b O=WAt Y4+WZ-C’A(ATA)Z 


=b’ AY + C’AZ—~C’AZ=BA' ¥ (3.7) 


Var (b’ ©) = Var (b’A* ¥) 
= b’ A"D (Y) (A*y’b 
=b’ A* oI (A*Yb 


W 


ofa) 4 “ 
2 [bt (AA) 
y 6 is unique and has the Minimum 
: sal s show Hae estimators. If possible, let gy be 
Finally, of all unbia 


ance less than b’At Y. 
4e) which has a varian 4 


jmato 
nother unbiased est 
a 


Now var (dy =dD (Yd 


=d old 
=o dd (3.9) 
Var (b' @) = Var (BA +1) 
=b'A'D(Y)(A') b 
=h’A* ol (A') b 
=0°b’ (A’A) b (3.10) 
(3.9) ~ (3.10) = 0° [d’d — b’ (A’A)* 8] 
=o (d-(A*) by (d-(A*y’ b) 20 (3.11) 
Sine dAO=b'O>dA=bh'>d =b' At 
R.HS. of (3.11) 
= 0° [dd—d'(A°Y b—b (A*) d+ b (A*) (A*)’ 5] 
=O Ld B UA (AY b—B (A) (AY b +B (AY) (A*Y by 
=0'[dd-v' (A’A)* b]= LHS, 
= Var (@'Y) - - Var (b’ 8) > 0, it follows 
Var (b 6)< $ Var ('Y), for all choice of d 
Thus, b’ i) is the most, fficie 
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[since E(Y)=A@ and tr (I-AA* 


model theory approach. 
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Sa aa = 


1 G-Inverse 


Le, d’d = b’ 
Finally, b’ © — 


Proof (ii) : To Show that 


o? = Y-A Oy (y_ -A®) 
(n—r) 


(3.14) 
where A’Y = A’4 @ are the normal €quations 
We have, 
BEBE) tena vone) (3.15) 
[as R.H.S. of (3.15)=y_4 O-AA*Y + ata Q 
=Y-A O-AAY+46 
=Y-AA*¥=y_46~ L.H.S.} 
Note: (J—4,4* ) is symmetric and idempotent matrix 
=> tr (J — AA") =) (l- AA) Sn) (3.16) P 
Now consider $ 
i) A ©)’ (Y- A©)} ir 
=E[(Y-A ey (1-AA*)? (Y-A ®)], by (3.15) ils 
—EIY =A OY GAA ep 9)] (3.17) Py 
[as (1— AA*) is an idempotent matrix] ‘ 
= 0° [ir (-AA*)]+E(Y_A ©) (I-AA*) E(Y-4 6) id 
[as E(Y’GY)=0° (tr G)+E(Y’) GE (Y), by (1.74)] c 


=O (n—r) 


J=n-r, by (3.16)] 


Thus an unbiased estimate of o° is given by (3.14). 


This proves Gauss—Markoy theorem rigourously by unified linear 


ISSION | 
Sox paid) 
“Ne 


ac BIBRARY )¢ 


4s 


e nimi 
oO ay nimise 
ar constrains unconditionally cal 

matrix 


, y-A 0 ates 
ge(¥-A e) if a Langrangian Multiplier 
) xl - , 

Phe: yA9t+0'AAO+2NBO 

=VY- i 
o pee 4004218 0 
=VY 


i 49/40 = 9 = 


} In other words, we have, .- 
! AA @=AY-BA 
; a ve 
‘ Now for any (Bn x m with p (B) =¢ say, we can ha 
: 
. ; 2 
‘ Alexn (A), xm (Qmxn my (Bm xan ( 19) 
« 


(3.19) when substituted in (3.18) gives 
AAQ=AY-A'ACA (3.20) 
i | . Premultiplying both sides of (3.20) by C ” we have 
if C’AAO=C’AY-C’AACA 
if = C’A'(Y-AQ)=C’AACA 
E | by WNEAAQ CA’ (Y- A) (3.21) 
ij Again from (3.18) we have 


A'(Y-A)=B") 


: (C’a’ACyt [AiY-Ao)} (3.22) 


(3.23) 
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G-Inverse 


43 
BUY a ie eae 6) 
=(Y~A @y (i —AAty (Y~A@) 
=(Y~A ey (7. 


ARYA Gy 
(i> AA*) being idempotent 


Again, (Y—A ©)’ (Y~A@) 


+y\, be Se 
* (A’Y’ [B’ (C’ A’AC)* C’ A‘ (Y-A®) 
Denote 1 term of (3.24) by J 


2" term of (3.24) by 17 
pad 3°" term of (3.24) by II] 


Note that J+ //7=(y_ Oy (A*y A’ (Y_A ©) 


=(Y—A ©) (AAty (Y-A®) 
=(Y-A ©)’ (AA’*) (Y-A®) 
=(Y-A®)' (Y-A@) 

= LHS. of (3.24) 


Now (Y-A 0) (J- AA*) (Y-A®) isa quadratic form of the type 
Z’BZ, where (Y—-A®)=Z and (I - AA*)=B, an idempotent matrix 
which is distributed as yo? with 


and [J + II] +] 


d.f=p(/-AA*)=n-r 
[for proof see Biswas, § (1993)] 


By similar reasoning I ~ x o with (r—t) degrees of freedom. 
where p (A)=r, p (B)=t andhas been reduced to zero by putting (3.23) 
in (3.24) and III ~ x” 6° with p (B) = t degrees of freedom. 

Since 


p (B) =p (B’)=p (AAC) <p (C' A'AC) 
=p [BA)* (BA*Y' 


ead of 
nd the 
Delhi), 


oona 
power 


ttute, 


names 
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yTION-REFERENCE: RAO, C.R.) 


projects every vector O into the 


ix which 
anned by the column vectors of 


ro jection 
si tor space SP 


Let P be aP a vec 


estimation space 1.€- 
A= (aq; Ay Am) 


Obviously. PA =A 


ved that P isa symmetric and idempotent matrix [Biswag 


It is pro 
§ (1993)]. 
Theorem 3.1 + 
idempotent. Convers 


projection matrices. 
Proof : Let P be a projection matrix which projects every vector in V,, (a 


linear manifold) in itself. 
py=Y if Ye Vy 


If P is a projection matrix then P is symmetric ang 
ely all symmetric and idempotent matrices are 


irX¥eV, #PX=¥ 


= ¥=PY=P (PX)=P°X 
Also Y=PX 
| = (P-P')X=0 
Hi = P=P" je, Pig idempotent 


itXe V, and Ye Vj, then 


2 is 
i X=¥+a vector |” V, 


» $= 
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=> (X—y)= ly 47 


a 


(X-. ry PX = 0 

i.e. (X- PXY PX = 
(X’ — xX’ P’) PX =() 

X’PX is x’ P’ Px =() 

X" (P= PP) x=9 


7 P= P’pP 
a P= P’p 
he. P=p 


— Pisasymmetric and idempotent mat 
ialrix 


Converse: If P is sy 


2 mmetric and id, 
matrix. d idempotent, th 


en P is a proje 


Let Y= PX and P isa symme 
ftric and idempotent matrix 
4 P= p2 é 
ay ,. ni 
Y= P’X = P (PX) = py 
= P is a transformation matrix whic 


space M projects VY € Va into the same 
Again if Y= PY=px as 
=> P(Y-X)=0 
P’ (Y—X)=0; since P=P’ 
i.e. column vector P” is 1" (Y— X) 
(Y—X) ' Y=PY 
(X-Y)1Y=PY 
i.e. X—Y is 1’ any vector in V, 
— X=Y+a vector lL’ V, 
= Y isa projection of X in V, 


“. Y= PX => P is a projection matrix. 
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Y-py\' 
_unbiased for b’ ©. a a 
whereas Y-PYe Ve 
We can write 
PY= a, 9 
(a, 
=A (3) 
using PY=A 0, Y-A Gisa vector 1’ y = 
ie. -A'(Y-A)=0 
A'Y=A’AQ 
O=(A’AY" A’Y if (A'AY" exists 
ae 
+2Cov[C’(I- P)y,c’py, i 
ae Now to obtain P 
PY) , 
: AO=A(A'Ay' A’y 
- or =AA’ ¥ 
Be (Y EY’) Pca A 
| | PY=AQ=A (AA) ’y 
~EM)E(Y)=0° 
or AAT Y¥ 
Theorem 3.2: C’PY has the minimum variance in the class of all 
___ unbiased estimators (BLUE). 
Proof: Let d’Y be unbiased for b’® 
‘a E(@Y)=b'0 
aon dA @=b'O holding V O 
dA=b' 
Also C’A=0' 
(C’-d)A=0 


A=[Q,, O, ..., O,,] is the vector space spanning estimation space. 
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5.1 CONCEPT OF A REGREssion FUNCTION 
Regression of Y on X, X, .. 
4’TA’C* 


ie 


demy 
. X, is defined as 


wwvan 
3 has 
E(YIX, X,..., X) (5.1) 
o where Y is the Criterion or Dependent variable and X;’s are predictor 
a ae A)'b variables. 
> of C” satisfies 


Similarly regression of X on Y is defined as ity of 
C’Y) is minimum EUR), Yoy nc Y,) 


(5.2) iva 
In defining regression of Y on X, (X, Xp, ..., X,)' are considered as 
non stochastic as they are given. Similarly for X on Y, (Y,, Y>, 


wwe VAY are 
taken as non stochstic. Estimation of regression line of ¥ on X =x given 
by the regression equation 


Y=B)+B, x (5.3) 


is same as fitting a straight line on the data of Y for a given value of the | 
I.v., say X = x. However, the estimation of the regression parameters are | 
not the same as that of a curve fitting when regression is non-linear, 


POT SPE ala 
rear e o 


ar 
panda? 
yexBr+¥ 


the O.L.S. of B given by B 
We have 


pa(x’Xy XY 


ii If D(y=ol 


otation as follows: 
ixn 


Reg, 


¢ 


SSI 
——__ Slon 
eee an aly. 


D ERRORS oF TH 
- _Heeasen . 
MATION" aRAMETER 
ON 


write the same as 
we row 
y ay 


(3.4) 


ty 
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x x 
x | 
n= (x, x) His 278 =3 | | 
i ieee eae oe" 
jig al ees 
x | || 
% (x; x) x (x; xy || = 
ss : i biszé ' 
4 
2 : - 
2 x; ; ~~ pa Xi y 
Bo nX (x —x)° x (x; 0 
A a ~ 
B, rxdy, Xx) 
nx aie 4; 
z (x; -x)? ZX (x,-—x | 
6 2. Xi Vi —n x \ 
= = 


As usual 
2 a 
A XL xe-nk ¥ Xi Y; 
Rea esaapersrprane 
ny (x; -— x) 


Standard error of By and B, are obtainable from 
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Cov (Bo Bi) = 5 we: 


git eee 
var (Bi) = 5 (x,- 9) 


[ n sé 


be employed. a a 
= §.S due to departure from regression = (y;— y,) 
; igh 
=E(%-Y+Y-I) 
=E (y,-)°42 629) -22 0;-y) 0 -») 


=E (,-y) +E 0)-9)°-220,-y) b; @- 9) 
=E (9-9) +2 (y;~-y) - 267 E (x, - x)" 


| 
| 
[since ¥ (y;—y) (x,—) = b, E(x, - 7° 
EG) 2) -20.G,,—)” 


=2()j~y)’-Z(y,~5)? 


(5.13) 


(3315) 


[since 0; ~ y) =b, (x; — x)] 
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y= y, += (, - 
a yy =S.S from the ry 
=Zy?_ & yi 

Also, = 0, = yy 


=S.8. due to regression 
= bi X (x, ~x) 


xj — x) (y;-y) 


= bl Bs y; =x = 


r(;-y) =SS. 


due to departure from regression 


5.3 ANOVA TABLE FoR THE 


TEST OF SIGNIFICANCE OF 
REGRESSION CoeEFric) 


Sum of Squares 


{28 -<eyi% 
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Regression 


ANOVA table is obtainable by incoy 


é Porat. 

of able 

Amore ni a ied ae of Yinto the table. It is c 
actor fo 


Now we can write 


ny an ye 
55 (bp) +55 (0) 10) =i EH Y— NEY + Ny 
=b, 2x yt+nyo-5, x) 


=b, 2x); + 49 Xy; 
Ly; 
= (bo, b,) 
| 2 XY; 
Hie =h'X’Y 


S.S. (Bo) + S.S (b; | bg) = S.S due to regression with 2 df. 


Hence, one can write the ANOVA table in matrix form as 


b’ xX’Y/2 


(Y’Y — b’ x’y) 
(n-2) 


aa ; 


alled S.s 
the correct” gente a 
if be toby. This a sum of Squares Mean 
: } Squares 
Hi Ey=0 ii 
i Zr y)/n] = (0 (ii)/| 
if ion (0) 10) by xin Os : £ 
} ares pe y2-()- @) =) a een 2 


Coie alg 61 
Let us define X 
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Y,=(1,X,) =X A anpower 
a I-A B (a Scalar) nstitute, 
B, ) ecades 
a rammes 
=B Xi Science 
We can write (5.21) ama at 
3 ‘hool of 
Var (Y,) = Var (Bo) + 2X, Cov ( ; ; ingham 
bee Bo, B 8 +X) Var (B,) acity of 
This quadratic form can be written as “ Sitor 
y awards 
F Var (i) Sei boxe librigh 
Var (B,) = (1, X,) bse (Bo, By) | | | | enter, 
ria A | indian 
Cov (By, By) Var (B,) | e4 11987, 
\pplied 
=X 0 (XX)! H=K! KK y 22 Tete 
| | k= Ag (XX) Xo (3.22) cience 
Again Multiple Correlation Ceofficient defined as ademy 
under 
R= S.S. due to regression nuvan 
= ——_____ — *Setssion n 
Total S.S. is has 
tae? ~ d nine 
PET _my, (5.23) se 
Y’Y—-ny” \.Phils 
sid ‘ersi 
is a measure of the extent of linear regression. ra 
New 
5.5 OTHER MISCELLANEOUS PROBLEMS IN  pro- 
and 
REGRESSION ANALYSIS wann 


9.9.1 “Extra Sum of Squares’ Principle 


Often it is a question while ascertaining a suitable regression model, 

whether a particular term or effect is necessary. Itis necessary in the sense 

that the sum of squares due to regression is increasing Lapse ad 
porating the specific term (or the Multiple Correlation Coe iain i: 
increasing on account of that). To be more specific, let us con 


model 


yy 


= Y’Y— ny? With a 


lag 
SSW -—SSR=Y’y_,, p2_ y’ 


Y+ Bp’ xy 
=’ xX’yY_p y? 


with (1 — l)—(n-k~lh=kdf 


Hence, the appropriate test is 
p=—BX'Y- nV, 
Y'Y-B’ X’¥/(n —k~1) 
Example 5.4: (An illustrative example) 


Given E(Y) =X B to be an appropriate mode] 


test My: CB=0 


| ie (1, a; if 9, 3, 16 8); B’ = (Bp, Bi, B., Bi 1) 


A pplic ation 


FG 4 4\- 
' ~ } 
(x’xy? yee 0 0 | 
=O .§ 0 | 
i -¢ | 
L4 ) O +} 

1/2 8) ~ 1/6 | 

bs 0 1/4 Q 0 

" ] /6 0 | S/& ] 

~172 8] 1/6 , 


1/2 Os = we 4. 
Shs. ee 
—1/6 0 1/6 
—1/2 0 1/6 
7/2 
alt 342 
S73 
2 
( 40 
Vv yy a 3/2 5/2 2) 
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eed ee 
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YY=(1579 3 7 8) 


= 278 
A 
SSR = Y’Y — B x’y— 278 — 275 5 _ 2.5 
Therefore, the equation for the null hypothesis 


My: CB=0 
B,, =0 (i) 
B; — B) =0 (it) 


B; - B.+B,, =0 73) 
2B, — 2B, + 93B); =0 (iv) 
This hypothesis can be expressed in a much 


Al : Bi, =0, B, = B= 
as the third and fourth e 


quations are line 
equations. Now the linear model 


More simpler Way as 
B, say 


ar Combinations of the first two 


Y= Bo +B, X, + By X,+B,, x? 
can be written as 


Y=Bo+B (x, + X)) 
= B=a and Z=X, +X, 


| 
5 
i ite 1\| 7 
oa Z'Y= 4 
“#00394 etl 4 
7 
8 
40° 
39 
/ 3 yl! 
and (Z’Zy'= 
3 13 
ee oa 
= 1/82 
—3 j 
&=(Z’Zy' zy 
| ee: 40 
= 1/82 
A3 7 }{ 39 
403 
= 1/82 
153 
40 
a Z’Y = 1/82 (403 (153) 
39 


= 269.3537 


SSW = Y’Y- W& Z’Y = 8.6463 


Now p=4, n=7, q=2, n-p=3 
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SSW = 8.6463 
SSR = 2.5 
SSW — SSR = 5.8463 = S.S due to hypothesis 


The appropriate test is now 


p_ (SSW = SSR)/q 
~  SSR/n=p 


= 3.5078 
Since F (2, 3, 0.95) =9.55 we do not reject Hp. 
As the original model was 
E(Y)=Bo+B, X, + B) X,+B,, x? 
and the hypothesis is not rejected, it implies 
B, =B)=B and B,, =0 | 
Therefore, a more reasonable model would be 


E(Y)=Bo+ B, (X, + X,). 


5.5.5 Weighted Least Squares 


In 1.2 we have seen that under Gauss—Aitken set up, /.e., under 
Y=XB+U 
with D (Y) =o" V instead of D (Y) = 07/, the estimate 


A 


Bea’ Vixy xv) 


0g on (5.50) | 


it even if the assumptions of homosedas- 
nple least squares estimate remain 
; to be the Minimum Variance 
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Re _ 
and D(U)=Vo" 
ig, OF % 0 
os 0 1/m, 0 0 
= ; ; o 
0 0 9 ive. 
a 
= diag (1/w, 1/@ ... 1/g \o 
ot V' =diag (a, W ... @,) 
B=(X’ Vv" xy! yyy 
@ 0 6 0 Vf 
a 0 @ O 8) i 
By X «1. X,) | 
Oo: 6 =o @, ji x, | 
@ 0 0 0 \fy\ 
0 ® 0 0 \ 
| 
(x, 5 Deere F | : : 
7°69 38 w y, 
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Special Cases 


Case I: Suppose Var (y;) = AX;, Le. Var (y;) 1s proportional to ) ke 


being the constant of proportinality. 
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E (1) = Bo + BiX 


Example 5. 


but the correct model is 
E(¥Y) = Bo + BiX + Bo X 
Let us consider the observations of Y corresponding to X=~ | o 


While estimating Bp and ), we propose to examine the n 
the estimates because of wrong formulation of the model, 


We have 
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E (Bo) = By + 4B, 
E (by) = B, + 73 


The estimates are in general biased. The extent of the bi 
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upon B, and fy. 


5.6 A PROBLEM OF MINIMUM MEAN 


SQUARE PREDICTION 
Letf (X) = £ (Y | X) be the equation. Suppose Y is predicted by f(X). Mean 
Square Error M.S.E is given by E[Y-#(XyJ’. The problem lies in 


minimising the mean square error w.r.t X. We can show that 


SQ) =E (VIX) =R (x) 


i MSE.=E[Y—f(¥1X)+f(¥1X) — yO} 
/ ’ =EIY-f(YAX) I+ ELF (YX) -f()P 
/ | PEIIY-f(YIX) FIX) -f()}] 
I MOELY-FUID} (VIX) —F(39)) 
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We have, R (X) =f (Y 1X), the Teg 
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M.S.E is Minimum Wher 
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This is known as Multiple Correlation Coeffici 
“lent 


Py (X), X,, -+1y X,) 
Also R (X)=f(Y1X,, x,, ves Xp) 
We have 
E(Y-E(yr =E(Y-f(YINP +E IX 


ene 2 
Oy = Oyx + Orv iy 


= variance due to departure from regression 
+ variance due to regression 


Correlation Ratio 
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_ Variance due to regression 


Total variance 
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covariane 
2 
E(y-a-B, % — By X2- + - B, Xp) 
= b+ B’C B - 2B’ 09 + Var (Y) (5.56) 


5.7.1 Optimum Choice for b and B given C, o, 
and Var (Y) 


Let b* and B’ be the optimum choice of b and f respectively. 


Put B =B"+6, then we have 
b? + B’CB - 2B’ 0) = b° + (B’ + 8)’ C (B" + 8) — 2 (B" + 8)’ o, 
=b’+B” CB°+8'CB'+B"CS 
+ 8'C5 - 28” op — 28’ oy 


If we put CB* = op and b’=0 the R.H.S of above equation be- 
comes 


b° +B” CB - 28” oy +8’ C5 > b2 4 B” CB" - 28” a, 


(since 8’Cé > 0) 


@ =£(Y)—- BE (X), the optimum value of o 
Therefore, the minimum value of 


Also 


E(¥-a-B, x; iy Ce =p, X,)° 


eVer()+ A cp" 28% o, 
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. . . 2 
irrespective of whether C“ = C™ or not. 


we may opt to test two types of 


In case of (i) we say that the regression model is same for both the 
Series while under (ii) we say that the two regression models are the same 
except for a change of origin. We proceed to develop a methodology 
fortesting both hypotheses. Let for the first and second series we denote 
the sample size and mean values as below : 
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Mean Values 


Head of 
3 and the 
so of Delhi 
Sample si2 4 1 x?) x 1) : Pou 
a u u 
n anpower 
= —(2) 
Q) y? *] ay Institute, 
n jlecades. 
40 the correct Sums of Squares and Sums of Products as rammes 
The defining ¢ ot a Science 
sO= (xf) =") (irk Hr Dv P= 1, 2 6 ama at 
fd chool of 
ft 
ae ungham 
il sf) = = OP -#) 0" -y") and Dacity of 
yj 
if E FS Pp ‘sitor in 
50 = bP (y-y A LI CAT ONS OF awards 
: 00 = tr 
} r 
Now employing the above results, the regression equations 


LINEAR ESTIMATION = 
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6.1 ANALYSIS OF VARIANCE (ANOVA) nd nir 

Often a Researcher, while comparing whether or not there exists real tia 

difference in the measurements of a certain attribute in the two popula- per 

tions draws samples from each population. For example, he may wish to ulty 

study a population of individuals with a certain disease who receive a 4, Ni 

standard treatment (or therapy) and another population of individuals with te p 

cat same disease, but receiving a new treatment. In this way he may plan to 32 

sas coe ie Sl compare the effect of k treatments (k > 2). In such situations, the inves- Ind 

tigator, instead of using two samples at a time, may wish to make a single 


test to find out whether various treatments have the same effects. This 
kind of problems lead to application of F-test with single, double and 
multiple classifications; with one or more observations (equal or unequal) 
in a cell. This leads to the problem of Analysis of Variance (ANOVA) 
where we obtain estimates of variance relating to several components, 
viz., between patients, between treatments, between hospitals and also 
interaction (if any) between them. All ascribable factors having been 
identified and the sums of squares relating to all relavent factors having 
| been worked out, the ratio of S.S due to treatments etc. are tested against 
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= Oy Q, +O OF... + % Oy (6.2) 
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E(Y=(Q, %..., O,) 


bn (6.3) 
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o,’ Y iy 0 A (6,5) 
, Y - 
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n, 9, 
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Zw | | me 
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Testing of Hypothesis Hy : 6; = @,=...=0,= Qo, Say 
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with (k- 1) d.f. which is same as (6.12). We can therefore construct 
ANOVA table for one way classification as follows: 


a ()/k — 1 = mst 
Ez T?/n- T’/n re (2) 


Ly -z T?/ni = (il) G)/n=k= mse 


Ay (Y-A@ 
" Y-A 0)’ (Y 
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6.1.2 ANOVA for Two Way Classification wit 


h One 
Observation Per Cell 
(Randomised Block Design) 
Here we have the model 
EQ) =U+ a; +7; (6.17) 
where Ll = overall effect 


a, = effect of the i" block 


1, = effect of the ;" treatment 
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Also normal equations are given by 
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0) T, In general, block and treatment Contrasts ae not « timakt : 
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i y j re (6.29) 
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fs It follows that first (6+v +1) vectors in A given above can be 
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i * P (Vq) S bv (6.42) 
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to test the hypotheses 
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x y, = constant Vi ) i=] Mi bs i) Institute, 
1 Jecad ; 
But ij Constant ee 
Block Contrasts Science 
: Q; — Oo) for i# i’. From (6.44), we haven — ae Jama at 
Let us first try to estimate (0; — a, Ve, = Sd Yi- XZ Fy) =9 chool of 
v ‘ ningham 
A A a 
i Wihst+r 2 tr > y, ee Lee pacity of 
Yioo = vr M+ vr 0, yay, 090 Yoro)/br = (Tj~%,) fisitor in 
er Er Re awards 
5 ‘ => 0/0 — Yovo) = (T;- T) ap ulbright 
ae a (6.50) ; 
Fon, = Yoo =vr (Q,; — ;) Se ral cat Vi ae Ni j I th d , me Center, 
ij ij n other words t ~ TIS estimable by Fie Yor. But the Preceding & Indian 
argument, every treatment ©ontrast can be estimat i % In 1987, 
| ed SI Jive . 
But given © y;=constant tions under given restric. Applied 
n 1991: 
A A ee | x : 
= te © ty) =0 | Interaction Contrasts Science 
ij ij cademy 
ee From (6.47) we have unde 
(Yioo — Yro9)/vr = (Q; — O,) bhuvar 
| T= Thr dit rt 4 oA, fas ha: 
nope eo 
= Yoo ~ Yoo) = (6; — 6) (6.49) | A. i tea nd niny 
{ Yr 0= PR + rOy + rt + rYy 2S ani 
. ’ M.Phil 
Since any other block contrasts are capable of being expressed asa : : mae ees iverst 
linear combination of 0; — O for different and 7’, every block contrasts By PO SO) br 8 cee “i ei) culty c 
S estimable under the Elven restrictions. It is obviously seen that the interaction Contrast is not estimable by h, Ney 
| y y Al te prc 
} yO ~ 47" ;'0: sO an 
Treatment Contrasts | _* a eo aa ed 
We have from (6.46) | Yio — Yr ro =r Yioo ~ Yoo) +r (¥ 070 — Yora) +7 Cy ~ Fp) 
b b | eS Yio Ve jo= (Yioo — Yoo) + (Yojo — Yoyo) + Wee) 
* 
Yyo=britr > Otbrt+r > 7 | gel hea ae ee es: 
i=] i= or (Vii — Ye i) =( V9 Tal Ying - ¥-00) — (Yow - oo) 
| (6.51) 
- : / 
if eee . A 
E Yyo= bri + ap O; + bro. +r oy i 
eer i=] 


By a sin 


umable under above restriction. 


nilar argument. it follows that interaction contrast is es- 


A 
sie = Co ¢ 
5.5 due to esumales (unrestricted) AY 
Now ». 


=E Gi+G,+t+%y) Yio 
ij 


= 2 (Yio/r) Yio 
1) 


S.S due to estimates = £ ( Vio/ r) 
a] 


with bv degrees of freedom, because p (A) = by. 
S.S due to error (unrestricted) 


=Y'Y-0’A’Y with (byr — bv) = by (r-1) df. 


= Ed ae © Yar with by (r—1) df. 
Lf, hy 


- To test interaction, BE) = 


Regress 


| i 
i & 
a 
A 
Ty (Yooo Yi00 bine Y;00 Yoio eee You Yio Rieke 
I A 
bi Ty 
‘ 
i Vit 
| : 
ii A 
H] You 
| = fh Yo + ZG Yoot Zt Yoot Z Yy Yio 
i j ar 


Yi2 =... = Vy =c, say) we have, 


ion Analy. 


i ee ltC~<COw”tSTS 


Yr.0) 


(6.52) 


(6.53) 
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ee 


h 
E(Y)=my 4+ by b; a) + et Ze Y 
f= j= ab i,j nit 


under Hi, 


FSm(e+y)+ b 0,4 5 
io | 


f=" 


fy 


Note that this model becom 


es the same as thatof R 
Design (RBD) with one observa _ 


tion per cell, 
S.S due ie estimates under H,=$.5 due to estimates (unrestricted) 
under R.B.D with r observations per cell 


domised Block 


= 2 ; 2 2 
a x Yigg/ vr ap Yoo/ br = Yo00/ byr 
J j : 


S.S due to interaction = §.5 due to H, 


= Yig/r- = Yioo/vr — X Yoo/br + Yooo/ byr 
i,J} t } 


(6.55) 


with (bv —-b-v+1)=(b-1) (v— 1) df. [as S.S due to hypothesis = §.S 
due to estimates (unrestricted) —S.S due to estimates under the 
hypothesis. ] 
S.S due to estimates — S.S due to estimates under H, 
=S$.S due to H, 


. S.S due to estimates (under H3) — S.S due to estimates under H, (given 
H, to be true) 


=S.S due to H, (given H; to be true) 
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Precisely, by the same way; 


to be true) 


= §.S due to H, when H; is true 


Yoo, » Yoo» Yoo Yoon 
Oe eae atin br 

ae 
= 5 ee with Me tB)~ F=(yv1) 45 
OF 


= S.S due to treatments under A; 


with (bvr—-1)=(n- 1) df. 


6.1.4 Some Critical Questions on the Appropriate 


Tests of ANOVA 
oe 


‘late test will be F = V,»/V, 
nd by (r~ 1) d.f. respectively. 


onent is found significant. Is there 
raged over all given treatments even 
etween blocks for a particular 


S.S due to estimates under H, — S.S due to estimates under H, (given 


8ression Analy. ; 
ISLS 


Yio Yoo, vith fe 8 vab 1) 4; 
x yr br Ge 
= §.S due to blocks under #7, (6.56) 


H, 


(6.57) 
Now under H; the usual additivity of the S.S holds 
. from (6.53), (6.55), (6.56) and (6.57) 
2 Yoo 
Total SS= 2 V,-——=T (6.58) 
ad bvr 
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aaa ae eee = es 


Here we require to test blocks wit 


h fixed Treatment: 
€.g. On,= Op, = .. : 


— OL, 


(iii) Is there difference 


particular block ? 


The appropriate test js Vi/Ve. 


(iv) Is there difference betwe 


(v) Is there difference betwe 


in the experiment 
nly ? 
The appropriate test is V,/V AR: 
Note : The difference between [ 
to be noted. In case there is n 
[(u1t) and (v)] are the same. 


(ii) and (iv)} or [(iii) and (v)} 


is important 
© Interaction, note that {( 


ii) and (iv)| or 


6.2 ANALYSIS OF COVARIANCE (ANCOVA) 


The technique of ANOVA enables us to c 
while Regression analysis technique is restricted to treat the factors 
quantitatively. Analysis of Covariance (ANCOVA) is considered to be an 
extension of ANOVA technique in the sense that here we consider some 
of the factors quantitatively while others are dealt qualitatively. The 


variables which are treated quantitatively are called Concomittant vari- 
ables. 


onsider the factors qualitatively 


Sometimes it happens that even after taking into account of all the 
plausible qualitative factors in a desi gn, the residual variance still yn 
significantly higher. For example, the variance of the treatment se 
Say Var (t;—1,) which otherwise could have been highly = mh 
shows insignificance when tested against the error variance which is 0 
very large. This makes the design inefficient. 
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introduce certain other factors which 
wine the residual variation. The tech 
erformed is known as Analysis of Co 


Hence, we require f 


Could be 
dealt quantitatively to b 


Nique by 


which the analysis 1S P ee 
(ANCOVA). 
6.2.1 The Problem of Choosing Concomittant 
I “Variables 7 — 
| The choice of Concomittant variables should be governed by the iis 
ing considerations. 


(i) The Concomittant variables x}, x, .. x, Should have a direct Cause 
and effect relationship (not ae the correlation) with 
dependent variable. The effect of association (linear) m 
measured by Multiple Correlation coefficient of Y on X1,x 


Y, the 
ay be 
2 ry Xp, 
i.e. Ry.» However, a high correlation does not nessa 
mean that the variables x), x, ...,x, should be considered Con 
comittant. 

() The Concomittant variables (each of x,, x5, ..., p) Should be inde- 
pendent of those factors which are treated qualitativel y: 
(ii!) X),X,...,X, Should be independent among themselve 


should be uncorrelated with the error com 
linear model. 


S and they 
ponent, say €, ... in a 
(i) Ibis advantageous to assume D (Y) = 67/ or y, 


on aie ns Vo 


’s (the observations 
» are apeally independent and hav 


€ constant variance 


; 
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ili 
5 v 
E(Y)=m+ x b 5 
fet Om Tyr s S&C, 
J r=} 
where 
0 
| 0 
1 | 
m=| | aL Yea 
i “nit Corresponding 
| to v unities 
0 
by x} 9) 
0 
1 | 
J treatment corre— Xe ~X 
1 Sponding to unities ei 
0 X+ (12) — X, 
fj : C.= 
0 
: te 
13> J’ treatment corre— ¥r (bv) — Xr (00) 


Sponding to unities 


] 
where Xr (00) = 5 Xx 
ij 


Hence, we write the model as E (Y)=A®0 


where A =[m, by, bo, .;.., Dy ty, ty 


++ ty Cy, Cp, «0.5 Cp) 
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denoted as below: 


> _ pare partial regression coefficients, Symbol , 
-=1,2,...,0 4 
where , 7 


Te 
a 


S.P. Matrix of S.P. Matrix of 
i S.$() | a -Yand y nee 
Source | a (4y, 22, =» Ap) and | My 2) 44, Ny, 
ah B. i 2. Bo, Ss By (B,, ),, ; 
Block " a 
7 7, Ty (7,5), } 
Treatinent 
E\, Ed En (E,.), y 
Error - = a 
ie i Gi, G2, Gyy (Gy), xp 
Tota | e oe. 
Db 2a we an 
Here, B,.=v X (%» 0 -—-,00)) Ow — Yoo) 
ie | 
x iis _— 
Ty =v Z (% i — Xe 00) (Vo; - Yoo); S=1, 2, ..., Pp 
j=l 


Ga Ss B,, Ea Ly =£, 


Gy = ZX (XH j- Xp 00) (i — Yoo) 
fe 


mechs 1, 2,2... 


(B,s)y xp is defined by 


ny = (X10 —X, 
i=] 


Similarly, (7, 
7, =b 
i 


and other symbols are similarly defined. Normal equ 
A'¥=A’A Q 


h 


X00) A¢6) 9 —X, (x) 00) 


rs)p xp 1S given by 
p 


Q (i 07 ~ %, 09)) (Xs) 97 - +s) 00) 


ations are given by 


ton Analy , 
SS 


Applications of Linear Ff 
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Stimation Te, hinij Ju 
ant Ne ru qu es 
ae } sa 
mm mb, in’ b f | 
Mo ‘ ; h Mm 7, 
‘ i b, m b, b, b’ b, b mM | m’ ¢ m 
a Pvt, b : b? 
x, bm b b 2 fie 
b, iy, by Kg °» Dr by fh by tb 
fe I 1 Dy ar - hi bi. 
Ve fee lobe & | 
2 hs 
by ; 
v im ty, l! b 
cy ok a + ree : aa 3 
pn cy b Che = ae ee 
4 t € c Cc 
Ss, | 
L ; , 
C, mM ¢ b, Soi Dn ag f 
| ; ct ¢ C 
A fh , A : | 
* (Ul O, a, T, ae a ee 
| Wie Sie) 
Also the elements Of A’A are; 
€ obtainable {rom the table below - 
i b; ! api 8 
of by \ s Rae Ss 
bj \ Sir) “ 
Y 4 8 p a 
: b | dir b bG x to) 
rd ) V Wy i9 — Xe 00) b Gy 97 —¥ yan) Z (Xr (i) — Xin op) & ( ) 
P (v,) = Rank of the estimation space 


=(b+v-1) for RBD 


Since p Concomittant y 
are (b+ +p -—1). 


ariables are independent, total parameters 


P (Vg) = (b+ + p-—1), because p concomittant variables are inde- 


pendent among themselves 


effects. Hence, the normal equations can be written as 


Vg 


P 
A A y 
M=VU+va-+ 


i tae ae ee 2) 
ae »y Ss (%9) 0 — 409 00)5 be 1 Qa eb 


p 
a> 


as well as independent of block and treatment 
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Yo =bU if j=! s ) oS Ta [H) ie ae eS Uy . of Dethi), 
: f jai MO ~ Yoo) ~ b« ; 2, Poona 
A i NOX. Ny ; 
ie RN Xs) 0-%s)00] @ +b X [x ute A iy 100) }} anpower 
E yp Gq ~%ep00) =” * Pes o~ % 001 % j=1 OF *s) 00) 4, FF @WH-XH0) (,, ~Sey ue Teens, 
Bay] tes reAys sh {s) 00) Sir (665 Jecades. 
es _— ra] 13} 
+ 2 Crip ~(r) 00) (Xs (i — Xs) 09) E, (6.61) Phas : pollen 
a . ae = rr o;r Fe p 
da, = Bey ae mee tig T8426 b=¢ yama at 
imation space isb+y~— ] + 3 gf. chool of 
As the rank of the estimati : P. P A ningham 
(because of p independent concomittant variables introduced), we impose = Sn By-T,= > E G.~B,.~T.) pacity of 
oe ; r=] oO fisitor in 
two restrictions, V1z., atin 
P Ibright 
b id = : - gnt, 
zr &,=0 and > %,=0 or E,, 2G Ess $=, 2, sau D Center, 
j=] I= ® Indian 
in 1987, 
to get particular solutions of Normal equations. This gives E,, Applied 
= Sis ant : E. n 1991; 
f= Yoo/bv = Yoo (6.62) E.=(, S2 --- &,) = | Science 
: : = sademy 
and Normal equation (6.59) gives ' E., } ‘aca 
Fi : bhuvar 
' p ZEC.E.=E fas ha; 
A A = Ss rs ry 
V(U+Q) = Yo —y “ §5 5) io — Xs) 00) nd'nins 
r= 
E... 3 ani 
P Phy M.Phil 
ies a = oa 2y = , iversit 
or 6,=Y,- % & Gn w ~ Fy 00) — Yop (6.63) (Er) =r Ga... &) culty 
s= : * 
E. h, Ne 
. te pr 
‘s : = a ae ; Ree ; 2 g ar 
? provided (E,,) exists. Now it remains to be shown that (Epxp is a bint 
25 y -Si IX, 1.€., to show that E =p. 
zi Cs 20) — ie (6.64) non-singular matrix, i.e P(E,.)=p 
: | | Also 
) in (6.61), we have a ee ae a 1 ee 
oe Se [i — 2, ay —F,q) +X PE 1,2, By J 
= = Fe XG — Xray — Xeon + Aran 
+ ° Gy Go = z LX (i) — X (0) — Xr) ~ Ar (00nd eri ~ FG) ~ Fron +, 00) 


tJ 
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ss ae 
QM em EF 


Now p (E,s)=P (61, S2---» G) =P (&), & ..., o) =p 


=  &,, is a non-singular matrix 


S.S due to estimates is © A’Y with (6+v+p 


— 1) df. From (6.62) 
(6.63) and (6.64), we have ; 


LL = Yoo/bv = Fog 
iui p i 
G, = (Yio — Yoo) - 2 2 Qs) i0 — X(s) 0) 
7 ‘ ae ae Pp .3 
oud G= q — Yoo) - ; F §s [so — X00) 
CED EP EF iy —X, 00) (Yj — Yon) 
ij 


“.  $.8 due to estimates 


= by (Yoo)? iz ee [¥io {Yio - Yoo) - - Gs A) io %(s) 00) }] 


P 
- ; a Qs) 0} - Xs) oo) }] + ee Ei G,y 
SS = 
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S.S due to €stimates ; 


= by (Fug)? 
( 00) EE Birt Tye Vi, +ZF G 
: "sr Ss sy 

SOV Yo) +38 on 
v ( 00) ig Bt +B 
5 ¥ vy) 
= Y : 2 
by (Yoo) + 5 E.y+ (Gy, - Evy) 
(6.66) 

with (b+v+p-— 1) df. 


S.S due to error = y’y _ A’y 
2 = A 

= 2 Yij— bv (Yoo? 8 E.y-Gyy+ Ey, 

with (bv—v—b-—p4 1) =(b~1) ~1)_pag 


.. S.S due to error = Gy- = e E 


¥~ Gyy+ Ey 
= Eyy~Ze, Ey 
with (6-1) (-1)-pag 


Testing of Hypothesis 
Ny: (§ =& =... =§,=0) 


S.S due to hypothesis Hy =S.S due to error under Hy -S.S due to error 
(unconditional) 
= Eyy— Eyy+E8, Ey 
= py é. Evy 
with (b — 1) (vV-1)-[(b- 1) (v-1)-pl=pd4. 
The appropriate test for Hy is 
»» ae E.y/p (6.67) 


i —l)-pi df. 
which is distributed as central F with p and ((b - 1) (v- 1) —-p) 
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waty= 7) 
To test Hy’: (1 = 7° =" v 


P 
b 

Ze, 
Fi ‘s, +2 ba,+mT+ vg 
, Model: £(0=™2 ied 


r=1 


b P 
=m(u+p+ » iad ee 


P * _ 
&, = (Vo - Yoo) - - Es (Xs) 10 — X(s) 00) 
where €,” is given by 2 E,. e= py T= 1,2,...,p 


5 (GZ, ze Bs) . i (G,, ae By) (6.68) 
s=! 


P * 
and 2 Gate & = Gy © Ey) (6.69) 
y=] 


€,” can be obtained from the solution of the above system of equations. 


S.S due to estimates under Hp’ 


a ce  _. oa = 
. WV (Yn) +y E OY ie Xo0) — = (%(s) 0 — Xs) 00) }] 


By -ZE," Gy 
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S.S due to h i 3 
YPothesis under Hj =85 due to ¢ 
due to error (unconditional) : oe ada. 
= Ty + Ey - 


Ze; (Ty + Esy) 
=Tyy~Z SS (Ts) + Esyy+ §s (Esy) 
with (by — b ~ p) — {(b 


~1) -1)-pj=v_par 
The appropriate test given by 


Ey+ Es (Esy) 


p= tw-ZEs (Ty + By) 45 


x 
Evy—ZEs (Egy /(b 


Es (Esy)\/(v _ 1) 
- 1) (v-1)-p} 
is distributed as Snedecor’ s F with (v~1) and (b 


-}) (V-l)-pdf. 
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; ‘ep Th dition, the re 

ich is linear in x. This proves that under the con : 
ips: is linear. Precisely, by a similar arguements, the regr 
- Xon Vis given by 


Reg 


PeSSton A 
n ; 
ss mea Shy 
7 “ta, 


2 
2 2 y~ Hw, -po, +=, 
4] “Al-P)) |e, 3, 6, 
i 3, 


1 é 
rie 
20 Oy *-[1, A 
re ae 


2n oO, 0; Vi-p’e 
2 
Oo, 
my a-p) [[o-w-e5te-u0 | 
1 4 
e 


The conditional mean Vixen iS, therefore 


o;. 
E(YIX=2)=Hy +P GMs) 


| o, 
—E(XIY=y)=y, +p 5, O- Hy) 


result does not hold good. However, if 


STession 
€ssion of 


Problems and Solutions 
i aa ase ee 


ee 
The constant regress 
i ; Se ES Higreet 
al variance Q on 1 (Mathai anq Pederzoh, of the condition. 
“I ’ ) 
prorat 2s Lat (X, Y) have the joint P-4.f. given} 


F(x, yy=4 if x>}y}. Q<,y 


<1 


=f Otherwise 
show that the regression of Y on X is linear 
is non-linear (Biswas, | 


99] ). whereas that between B on ¥ 


Behave s (xb y) x I/(1 1 yy, O<y<] 
because, f(xly =f&y) _ 1 


i nt OSes 
f (y) (1 —y)’ J sa i 


f@,y) 1 


A ain, ) | y-f& giias 
: FO *) f (x) 2x’ ¥i<g 


=> E(Y1x)= | 


wa 


5 dy =0 


i.e. Regression of Y on X is the line y =0, the X-axis, whereas 


t 

1 
E(X\y)= | - xdx 
0 


« 


| 
=—— ; 0<y< 
21-y e 


and is equal to J , 


: xdx 
ae 


Head of 
3 and the 
of Delhi), 
3, Poona 
anpower 
institute, 
Jecades. 
jrammes 
Science 
vama at 
chool of 
ningham 
pacity of 
fisttor in 
awards 
ulbright, 
Center, 
& indian 
in 1987 
Appliec 
n 1991 
Scienc 


Problem 


but 


where 


Problems and Solutions 
————— 


hers. 
EID =Z 7 4+lyD 


shich is non-linear. 
sa 3: Lek (YIX=y Hat B(x; -*) 


I o cov ()j, y;) 
WwW; 


homoscedastic variance as it arises in consumption and expenditure 
non- eee | 
its aiven by different social or economic strata). Th ain the Ss a 
to regression and total S.S. 


The weighted S.S 


Ef wily Ge 8G,-»)? 


Minimising Q w.r.t.a@ and B we have 


S.S due to regression 


BZ wy G3 


= A2 
=) £ Ni Wi (x, xy 
i 
k on, 
Total S.S =i ae w, y; (2 BD 
imfjaj ‘ Zee 
t 
By 
hi New: (x 
R*= — meena! ie 
kon ane 


which is the square of Multiple Correl 
Problem 4: To test Hy: 


(7) Bi=b, Vi=1,2,...,p 
(ii) B;=0 against 8,40, ;- 1,2. 


(ii) Bi)=B,=...=8, =, 
(iv) B,+B,+. 


voy 


mm p= B° forsomei=1,2,_ p 
(vi) A few coefficients 


(vii) Equality of two regression lines. 


Theoretical Background (An Alternative Tre 


The background for the testing of above hy 


be the multiple regression line of Y on X,, X, 


” observation 1, 7, 4:7) 


y= at B (xX, —X)) 15 B, (X) — Xy) + Pei B, (4, -3,) +; 
Where E(e))=0, Var(€))=0°, Cov (€,€) =0 


Minimising residual S.S w.r.t. ©, By Ba, os 


equations, 


ation Coefficie 


-- +B, =C, a given constant 


in the multiple regression model are zero 


potheses are as follows: 


Let E(Y)=a+8, (x, ~%)) + & (e+ ot fe 


Be X,. We can write the 


B,, we get the normal 
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Regressio 


n Analy; ‘ 
esl eT 


7 for B; Problems and Solutions 

Hence, B; is unbiased ic ac Head of 
P 35 Da ec 129 3 and tt 2 

, il ss U bi d . a ela of Delhi 
o 5 RG Sik An npiase Estim ot Delhi), 
Cov (B;, B) = J=1 k=l wd h 2 ate of o Poona 
Consider the multiple regression of y on X, anpower 
ee td a the observed and expected eI We i i institute 
=0 zr -* x sl! Sip Values (on the b; P ALY) and Y, be aE 


corresponding to the /'” ASiS Of the linear 


4 


a obse es 
k=l l=1 : vation. d= ] es .n) Then re “SI€ssion) rary es 
=, 2 n Science 
= o , x O-Y)= x Ly) — Fir 8 ( ; ama at 
ball = é PAE) a ae a er chool af 
ae ij 2 p Xpt- x,)) “EIO! OF 
s76) p Al Th 
(2) - ingharm 
; = > pacity of 
i = 1 for k = I = xy (y, ra yy mE P a A 4 cade : J 
[because = SSK =| hs B; 2 1~Y) (x5) ha , 
zeal J aware 
= 0 otherwise] " p f ; ulbrigh 
Fe i y es = Cente: 
Var (8) =s" o 2 ea i-X) (xy x)) 2 India 
off = si? 01987 
Thus, the dispersion matrix of p; = 5° 0 3 = 5 ( Pp n Applied 
, y die a 
) aN I y) z = B, bs (x, -x) n 1991 
, a orgs A rat eg Science 
Sampling Distribution of ye cake sademy 
A : . + 2 BiB = (i-% o under 
Wehave, B;=s' g, +5” a+... ae le Ray fey et PMD bined 
: = ae fas has 
If €,~N (0, 0, B, is a linear combination of the normally dis- a» pa) =0) nd nine 
tributed variables. 2s and 
; : 
A A iS ay - si os eee M.Phils 
E(B) =B;, Var (B,)=s" 07 and Cov (B;, B) = s/o? 3 ed Oi-y-2 E Bg iversity 
Jdieedinntac i ies ee 2 3 culty of 
oint distribution of B;'s is multivariate Normal of the form aa ay a: h, New 
>. ah B; Be (xi - x) (Xj — X)) te pro- 
4 ¢ til as if g and 
“id — (p- B) (s”) (B - B) 2 : Indian 
ar = (y-y) tie. Bi aj+ ps B; (xj) -2,) (, ~x)EB, 
! ie * = j=] B J : j 
Sil 2g? hence : 
{Since we have from normal equations 
j ye 1-1 oR A a i Se 
( al Bi) (s’) (B; - B;) (4) B.z (xj — X)) (Xj1- X)) = p> (x1 - X) r= 8) 


l I 


z, (y,-y) -2 5 Bjgi+ 2 z Bg) 


Eg | j=) 


i sorb B, ... B, 
§ due to esuma 
AY, Bg 2S. 
ny? S.S due to estimate of a 


by ye Total SS 


Hence, R.H.S. of (5) is the S.S due to errors 


Next we have 
y= at B; (x-*) + --- + B, pi ~ X,) +e, 
y=y+B, («17 -%}) +... +B, (pi — X,) 
y-Y=a-yt(B; ~ B,) (%)-X1) +... 4 (B, Bs B,) ( 


y=ate > a-y=-e 


Again 


AS y~Y%=- (6; - By) (X1;-%) = ite (ms a B,) es 


Los E [(€;-€)- (B,-B,) (x) -%)P 


*pt—%p)) +e, 


pl-X)+e,—eE 


(9) 


Problems and Solutions 


An A : A 
E| = (B,-8, =f, —— 
and | Bi) (B; Bi) *1 |= (Coy (b by) 
- Baar 
(5) oe 
= ow pie ee 
(6) vn ‘i 
(7) tifa 
Thus si 
E[Z0/-Y)1=E [xy _> 
- O.-Y, [2 ~y) -Z big) 


SE ee ev = ; 
J €)"7] | E G-6) (B~B) 5, 


=(n-1)0°-po 


=W~p=1)¢ 
(11) 
x Qi-y)-E bg, 
a cH 
(n-p-1) (12) 
is an unbiased estimate of o”. It is also to be noted that we did not make 
use of the assumption of normality of the dependent Variable for the 
purpose of estimation. 


Testing of Linear Hypothesis 


n —% P A A 
X ()-¥)= E (ei-ey- E 6-8) G-B)s, 


iLj=l 


nA ae * 
Now & (€)-¢)~y’o° with (n-I)df. (13) 
t= 


> and thea 
of Deihi), 


1, Poona 


x B i 14 
oa (BiB) (B;-B)) 5, ~x° o with p df. (14) 
Lj= 
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Riese 5, 


Under oy, 5B 


<<< llc 
j32 


eee: independent 
ae d (6, - Bd are inaependent. 
Iso (€;- ©) 
Als ; ov 3 
n 2. © O-y) - ~ Bia: 
Thus, = ers i a 
{= 


if ae 4 4 A 
2 meee x A, ~ a. 
at * ij=l B; Bi) (B; = B,) Sj 


J=1 


~x7o° with (n—-1-p) df. 


(1S) 


d is independent of any hypothesis concerning the parameters 
and | 


Specific Hypothesis 
(@) Hy: B= 4 €=1 2 p) 


P ga 
Under Hy, © (b;- B;) (b rs Bi) Si ~X GO with p df. 
ij=l 


Again 
/ 


=] I= 


unrestricted error sum of squares 


P 
= @-B) ()-B)/p 
F= iJ=1 


n cai Pp 
2 Oy-y- Z bg/(n-p-1) 
= , 1 


(= 


is Snedecor’s F with p and (n- P-1) df. If the calculated value of 
F > 100 @% tabular value of F with p and (n-p—1) df., then Hp is 


reyected. Otherwise we have no evidence to reject the same. 


(ii) A, 7B =0 ((=1, 2, — 


A ja 
(Bisy= B; i~X° O with p df, by (14) 


: P 
ra ee ee 
D (y=3)— = bgi~X O with (n—p-1) df. is the 


Problems and Solutions 
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z = Pp Science 
z (y, y) y iG ama ai 
et y) “~ Pi 2/(n p ; 
i=} 1) Snool of 
is the appropriate test for H, se 
a pacity 
AN — 4 Sitor 
7 ANOY A TABLE OF 1 HE TEST awa 
Source of aie) — © ee ibrig 
iat } Me ¢ Cor 
Variation acy Fe Squares Me an Squar ; ve ¢ 
f Se Eiror + india 
due to p | 5 Cae = =a | n 198 
by, bo, .--» Bp | 2 bj ~ Dis Ee 
= Applie 
D 
| n 1991 
Error n—-p-1l}| Z¢ y =E 5 z (y; yy Science 
| 5 Pay Ee eal Meeks sader 
Total n= | Ste oe et — vi nde 
ee = f onu Var 
re Mee nce eS fas na 
(iii) Hy: 6, =$2.=...=B,=B nd nine 
“Ss and 
We have, under Hp, r Phil 
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‘ iversity 
g= XZ (7-H) culty of 
= h, New 
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and y= a+ B [(x)-X)) +... +(ey-E)] +e g and 


oH 
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3 
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indian 


Re 


vir (8) = 8" 8 

pam s2 Piss, from (12) 
s (n-p-1) 

f , B, - BP 

ft Thus tHE (1-9) - 2B, 8/n - p - 1) 


distributed as Student's ¢ with (n —p — 1) d.f. Alternatively, 
is dis 


(B,- BY)°/1 
SP eG -2 B; gi]/(2 - p — 1) 


is Snedecor’s F with 1 and (n-p- 1) d.f. 
(vi) Hy: A few coefficients in the multiple regression model are zer, 
We have the regression model 


y= 0+ B, (x,-%1) + eee +); (41 -%,) + €; 


n model is given by 


eH 


B, respectively, then 


Ce 
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Problems and Solutions 


Thus the S.S due to hypothesis = 


S.S due to ror under Hy 


~ Unrestricted § S 
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with (n — p — 1) df. 


is distributed as Snedecor’s F with (p— gq) and (n— p-\)jdf. 


(vii) Testing Equality of two regression lines 


Suppose we have two regression lines given as follows: 

Y= al + BY? (ay x1) +. + Bp (x-ay +e)" 

and yO) = at) + BP (x1 — XP) + «+ BO (Xm 2 ) +E 
OLS gives the two estimated regression equations as 


mes, A A 1) 
yf? = + BYP Gy—H) +... + By? Gi) 


oa A 2 A(2) or. 
=F 4 BP (ei BP + a + BP Cm) 
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The normal equations corresponding to the 


first e 
sion is given by 


Stimateg Te 


Teg. 
A(1) (1) Al) a 
ah = BY sf) +... + BE Sd | 
and the corresponding error S.S is 
ny oe 
-(1)\2 Tati 2 
EO = YY — E Bi? gl ~ 4? 
1 74 21) 


with (n, — p — 1) d.f. Similarly, the normal €quations co 
second estimated regression is given by 


"TeSPonding tO the 


2)_ A(2) .2) R(2) (2) 
gf = Bi Si] +... +B, Sip 


(22) 
with the corresponding error S.S as 
eee fb) 2» 
2 Om -¥ YE BP gi? ~ yo 
m=! i=l (23) 
with (2 —p — 1) d.f. Since the two S.S are independent 
Ay ny 
: a> 4 
EP | m=] 
£ P 
i z po? gs pe g?) ~ 20? (24) 
i i=] 


-l)= (n; + Ny — 2p —2) cf. 


4 equality of two regression equations, it 
1, 2,...,my; m=1, 2, ... Ny) belong to the 


(25) 


Problems and Solutions 


Since (24) and (31) are independent, 


ny 


= 
Ri-Ri= = Orvis 
I= i 


+n, 


m=\ 


ny 
Rs 1) M2 
E Big 2 Gi’-¥) 
=| f= 


t=1 


Po 
2) =Qh?2 M4 EB Bi 
py On -y yrs Br 8 jak 
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PRA 
= Ep gi+ . B? @ 


{=I f=1 


es 33), 
a (yl a I2)2 s 
(ny +) 0 ) ~x? 9 i 
: (32 
with (p + 1) d.f. under Ho me 
a, +N 
2 Oy -y)? - z Gf? yy? _ 
a . i ye 
ae oon 7)? 


n, +n; nts ae y? oe 
= z Ve | Gy)? - 
i=1 (1) +N) fe 1) ny (yDy2 


Ai, 
2) 2 ad 
~ 62)? +m)? 


m= 

ae th) 5 (2))2 

— x pe fy () , A, 

v2 (7) + Ay) oy) = 5 (y?) 2 
f=] | m ) 


a a 
+n, (yy? 4 ny (y)? 


= =) ca 
my s(2))2 a 
= ( 1) +My} ) +n; (1, +n) (y) 


ees 
(1) +n) pe 6"y 


(my + np) 


yll)y2 ze 
) + ny ny (y)? 


Now from (24) and (32) 


Fa Ro+ 
Rin +h) — ~2p- 2 


with (nj + ny — p~2)~(n, +n)-2p-2) =pdf 
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Problems and Solutions — 
—————— 
_(RO- Ri) m+ = P= 2) 
(p+ 1) Ri 


is distributed as Snedecor’s F with (p + }) and (n; + %2- 2p - 2) degrees 


of freedom. | 
Remarks : The above testing procedure is adopted when we require to 


test the hypothesis 
Ho : a? a5 a”, pi? <= bY... ; PRS = bo 
when the hypothesis is simply 


, (1) _ al2) (ly) — a) 
Ho : Bi’ =Bi + Pp = Pp 


However, 


i.e. the intercepts of the two regression equations are not tested, the normal 
ficients are obtained by adding 


equations for estimating the regression coef 
(20) and (22). 1.e., 


(yy (1) (2), A (1) 2) 
g; =, (Si) + Si] y+...+ 8, Sin + Sip ) 


S.S due to estimate under H,’ is 


z B, (g\” ee gi) 


Thus under Hy’, we have 
E (yf? — yO)? 4 : (2) _ (2,2 A 2 2 
FOP + Z Of —JP- E Aa +eM=— |» 
~xX° O with (n, +n) -p—2) df. 
Also from (24) 
a ee es 
~X° Oo with (n, +n, -2p—2) df. 


R- Ri ~¥ 0° 


_\a 
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R3— R?/p 


F= 
Hence, R? (ny +n 


~ 2p — 2) 


is the appropriate statistic for testing Ho’ whic 
cor’s F with p and (n, + nz — 2p — 2) df. 


Problem 5: R.A. Fisher gives the following data fo, the 
longitude (X;), latitude (X3), and altitude (X;) in Yield (y) bane of 
sample size 186. Denoting the small letters as deviation Pets Ae On g 
of the corresponding variables, we have © Means 


b is distrip 
uted as ¢ 
Nede. 


Ly? = 1786.6 E x? = 1934] 


me x; y= 1137.4 z x) 42 =— 772.2 


Y xy y= 592.9 X x5 = 2889.5 
XZ x; y= 891.8 ZX XxX, x, =924 ] 
Y x3 = 1750.8 X x) x3 = 119.6 
() Estimate the Partial regression coeffi 


cients in the data and lest their 
significances, 


Bi +B.+B, =0.25 


line ‘Tegression of Y on X,, X2 and X; is given 


‘xy f 2) +b; x, 


itted since the variables have been 
Means, The Problem is to minimise 
42 -b3 x3) 


Problems and Solutions 


Thus, we can either fing Out the inve 
postmultiply the same with 


“bY: Loy, and 


: by pivotal conden. 
30) gives the solutio 


in rows (20) and {10 
b» and by are given by 


3 directly 
» ANE Pivotal row ( 
and by substituting the value of b3 and b, 


0 for b, 
tively, row (10’) is obtained, Thus, b,, 


) Tespec- 


b; 1234 
by |=| 3822 | 
b, 6817 


Table 1: Method of Pivotal Condensation for the Determination of inverse of 
S.P. Matrix and the Solution of Normal Equations in Connection with 


the Estimation of Partial Regression Coefficients 


Se Eran ner 
Unit Matrix for the Inverse 1 (1)+03) 
QB) 


y 0| 0| 2087.0 
i 1) 0 2379 
5 0 11 27955 
00052| 0| o| 1.079054 
39923 | 0| 
= a 0 | 1798.346 
0001551 000387 0| 1.189815 
150.2| - 55349) -.18907 1} 1217.06 


22| 1.00023 
= 00045} - 00016] 000822) : 
00024} 00042} - 00016) 1.0005 


ee 
00083} .00024| -.000455) 1.0006 
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by = .3822 and b, = 1234 takin 
co , & the last fae 
Column (2) in reverse order Similarly, inverse of the Sp it 
-00083 00024 ~ 000455 
-00024 00042 — .00016 
— 00045 — .00016 000822 
1S obtainable from Column (3) while taking the Matrix in reverse order 
Alternat; vely, the inverse of the SP Matrix by pivotal Condensation 
aS seen from Ta e-1 is 
00083] 000240 _ 000455 
-00024 000416 _ 000156 
—~ .000455 .000155 000822 
This matrix when Postmultiplied by 
Zaxy 1137.4 Q; 
z= XY = 592.9 S Q, 


Problems and Solutions 


145 
[iz 


42.3785 With 3; 
(i.e. Hy: By = B = B, = 


0, is discrediteq) 
The multiple Correlation R js Bive 


pea 21 Q, + b, Q,+b, QO, 


at 


n by 


a en 


TSS. 


(ii) Testing the equality of Fegression cog 


If b is the least squares estim 


and 1&9 118 thus signitican 
999, leve} Of Significan. 


fficients 


ate of the Common value of the regression 
coefficients, B,, B», B,, then 


ea Q, *& +Q, 
sa 


J 


1137.4 + 592.9 + g91 
~ 1934] -~2(7 


72.2) +2 (119.6) 


= 0.3684 
The S.S. due to b = (.3684) 


Source of Variation 
Due to b 
Deviation from equality 
of regressiion coeffts. 

— Due to by, by, b; 


Conclusion : Ho is discredited 


ii) To test B, =0 


We have 


ANOVA 


965.9816 
121.8538 


+ 1750.8 + 2889.5 


(Q) + Q) + Q;) =965.98164 
Table 3: 


60.9269 | 110.4235 


362.6118 | 


0.5518 
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; _, {000831 000240 ~ 000455 
i (S)7 = .000240 000416 ~ .000156 
~ 000455 -.000156 000823 


Var (b,) = 000831 ¢2 


= .000831 x 3.8384 = 0.00319 
b, =0.6817 


0.6817 
ltl= = 
000831 x 3.8384 a 


with (185 — 3) = 182 df Since sam 
compared by the standardised n 
considered to be signifi 


ple size is very large this result can be 
ormal deviates i.e., a value of t> 1.96 is 
cant at 5% level of significance. 


(iv) To test B, + B, + B; = 0.25 
We have 


Var (b, + by + b3) => slo? 


= [000831 + 2 (.00024) — 2 (.000455) 


+ .000416 — 2 (.000156) + 000822] 3.8384 
= .00509 
Hence, under Hy 


by by + by 20,25 
f 00509 


is distributed as Student’s 1 with 182 d.f. As before, 
is very large, this test is eq 
‘Ay is rejected. 
Problem 6 


since the sample size 
uivalent to standard normal deviate and hence, 


+ Givenn=40 © x =159.7 ¥ x,=211 ¥ x2 = 846.75 
95 x7 = 144461 Sy) = 209.148 Sy, = 2338.733 
75a=0. b= 0.070757 where xy = a + bx,, Examine 
: on of w ( of dry jute fibre (x9) on the weight of 


en 


= 


Problems and Solutions 


it 


A ME 
S.S. due to departure from the line of regress 
SION 
Xo = 25830 4 0.070757 X, 
is given by 
Roo = Sop ~ Do, 
= 209.148 _ (.070757) (2338.733) 
= 43.6663 
seh (n— 2) = (40-2) = 3g df. 


Now considering the first FESTEssion equation 
Xo =0.25+0,] X, 


S.S. due to departure from linearity 


= % (X)-0.25~0.1 x,y 


X X> + 0.0625 +0.01 5 X} -0.50E x, 


+0.05 EX, -0.02 5.x, x, 
= 846.75 + 0625 + 01 (14446 


+ .05 (2111)-02 (10766.95) 
= 166.17 


)— 5 (159.7) 


To test whether X, = 0.25+0.1 X, can be taken as the regression 
equation, we take 


_ (166.17 - 43,6663)/2 
een y 7 aa 


= 53.304 


whichis a variance ratio with 2 and 38 d.f. and is highly siguiieen Hence, 
Xq = 0.25 + 0.1 X, cannot be accepted as the line of regression. 
ssi ation. 
Next consider the suitability of Xj = CX; as the oe pi 
sas 
This means that whether by dropping © from Xo : ts i pon 
Statistically accepted as regression equation. We tes 


Hy: a=0. 


il 
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Now Consider the least 
ce. ae 4St squares Tegression of Xo on ae 
lb Le, 
Let So = Standard Error of od 
Then = VWur +21/S,,]? 
Roo/(n a 2) 
= 0.3543 
The statistic ; obtained to fest = 0 js given by 
=(a-— 0)/So = -25830/.3543 
= 0.724 


with (40 — 2) = 38 df. is not signifi 
Xo = CX, fo as 


r some suitable C is te 
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The Correlation Tatio 


F=M -P)/(k_4 

Q-1Wy/(n— Ky 

where "=157 and k=6 
=> 


F = 0.464 with 4 and 1Saf 
This is not Significant at 53% level whi 
: ch shows 
assumption Cannot be ruled out. pea — 
Problem 8: Confidence limits f 
cients 


OF Tegression and Correlation coeffi- 


y=73259 


§,= 130.91 
r=0.7928 b,,=0.7007 and b,=0.8971 


5; = 106.8 


Obtain the confidence limits for the regression coefficient of ¥ on X and 
the population correlation coefficient. 


95% confidence limits for the regression coefficient is given by 


3 1/2 
(n= 2) 2 (4-2) 
Z(y;-Y¥ 


by, £ tos 
and E(y;- Y,)’ =S.S. due to regression of ¥ on X 
= NS? (1-1) 
¥ (y,- PF = 46 (120.9 [1 - (0.7928)"] 
= 249807.81 
(n — 2) E (x, = 106.8 x 46 (46-2) 
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= 216163.2 


(m — 2) E (x;-x)* 
eg spree 20862 
(n—2)Z(x%;-x? | 


56,;- yy? = 1/V0.8652 = 1.075 


confidence limits of b,, is given by 0.7007 + 2.02 (1 0750) 


=- 1.47008, 2.8723 
te. P{- 1.47008 <b,, < 2.8723] = 0,95 


Confidence limits for the Correlati 
Since the distribution of r ev 
large sample approximation, it signi ea 
Therefore, we employ the tran 


on Coefficient 


sformation 
BaF l+r 
ne 5 log, = 


which is approximately normally distributed with standard deviation 


ea _ (Z-s) 
5, Vn—3) Then +t= 5, 


_ where . S. =S.E. (Z)ands=E (Z), is distributed as standard normal deviate 


$ t with infinite d.f.) giving the upper and lower confidence 


back to value of r, we get the confidence 
in these tests arises from assuming the 
table of normal deviate will give the 
s procedure we get 
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Z=+ Nog 1+0.7928 ®, Poona 
Vung Nanpower 
: Institute, 
tes £.07893 decades, 
; grammes 
me S,=- RS =0 Science 
. 46,3 ~9-1525 ibama a 
nate school ¢ 
1= %— to og Ss, minghar 
a ‘pacity ; 
1.07893 ~ 1 96 y 0.152499 Visitor 
= 0.78003 | awar, 
Also if Se ee “ulbrig| 
r= lower limit of r Corresponding to L Cent 
lL, = lower limit of z 1 Indi 
: in 19) 
then | oF l+r; ’ Appl 
Med OS in 19 
Scie 
1 l+r, cade 
or OER | 
‘ibn 
=> r= 0.65272 Nas 
Ba and 
Similarly, L=Z+ toos S, bsg 
= 1.07893 + 1.96 x 0.152499 a 
viv 
= 1.37783 1C\ 
th 
= 1 log 2 wu 
2 : f= Up 
i eee : a: r) 
where r,, = upper confidence limit of r corresponding to /, viz., the upper 
confidence limit of z 
1+r, 
or 2.75566 = log, = 2.30259 
u 
=> r,, = 0.88046 


Thus, Pr [0.65272 < p $ 0.88046 | r|=0.95 
r= 0.7928. 


where 


in) 


16. 


10. 
1k. 


eo 
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STATISTICAL TECHNIQUES OF MANPOWER 
PLANNING AND FORECASTING 


This is a text on ‘Statistical Techniques of Manpower Planning and Fore castir a - 
exhaustive coverage which can be used by an average Practitioner in hi a 
manpower with very limited Mathematical or statistical background. s 

The solutions to varieties of manpower problems using right from the elem 
concepts of Probability to most sophisticated techaiques of Multivariate Ar. 
Stochastic Processes or Time Series Analysis have numerically been ili. - 
from the very grass root level. ee. 

Every theory has been illustrated by live problems on manpower - nv 
applications of almost all conceivable Statistical techniques have duly 
exemplified numerically for the understanding of a practitioner ce 

A full chapter on the techniques of Population projection vis-a-vis Labou 
projection; Live examples of Manpower Demand, Supply have been appen: 
applied in National planning. A full chapter dealing with the methodology of Ax. ; 
and Wastage using a recently developed technique of Cox's Partial likelihood is... 
a unique feature of the book. The book can also be used as a practical handboos 


of statistics for graduate students of statistics. 


APPLIED STOCHASTIC-PROCESSES 


A Biostatistical and Population Oriented Approach 


The basic motivation of the book is to provide the background of the theoretical 
techniques of Applied Stochastic Processes as presented in the first six chaptors 
of the Text. The remaining chapters are devoted to the applications of the same in 
Stochastic Theory of Epidemics, Clinical Drug Trials, Demographic Models on 
Fertility, Mortality and Population Growth, Survival and Competing Risk Theory and 
Stochastic Process in Genetics. The book is a unique one in its coverage of several 
theoretical techniques of Stochastic Processes: as Random Walk, Markov, Non 
Markov and Renewal Processes, Theory of Geiger Muller Counter, Palm Probability 
and Martingales Theory. It is equally unique and Novel in its coverage for the first 
time a host of areas of Biostatistics as Survival and Competing Risk Theory, Theory 
of Epidemics (both classical and carrier borne), Clinical Drug Trials and Population 
Genetics, inclusive of Stochastic Population models as fertile fields of applications 
for the techniques of Applied Stochastic Processes. 

The book is primarily meant for Post-graduate and higher level undergraduate 
student in Statistics, Biostatistics, Applied Probability with emphasis on Biosystems. 
The background required for the reading of the Text in Elementary Probability 
Theory, advanced Calculus, Elements of Statistical Methodology and Statistical 
Inference. The entire course in the book can be covered in two semesters. 
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